Artificial intelligence (AI) is opening new possibilities for biomedical research, particularly in the field of brain health. AI shows promise for expanding what’s possible, from analyzing brain scans to expanding participation in studies of cognitive decline. At the same time, ensuring these technologies produce accurate, reliable results across populations remains a challenge.
Presentations by researchers from The University of Texas at San Antonio at the Neurepiomics Conference and International Stroke Genetics Consortium annual meeting in early October showcased efforts to adapt imaging tools to different research datasets and ways to improve AI model accuracy. A broader perspective of AI from UT San Antonio researcher Juan B. Gutiérrez, PhD, mathematics professor in the College of Sciences, emphasized the importance of verifying AI-generated results and protecting scientific integrity.
“We are learning as we break things, and things fall apart. These technologies came to market way before we could develop the social practices or the ethical frameworks to use them and discover their true potential,” Gutierrez said of the challenges of trying to use AI technology while learning its limitations.
Improving AI fairness across populations
During the Neurepiomics Conference, Ngoc-Huynh Ho, PhD, instructor in the Department of Radiology at the Joe R. and Teresa Lozano Long School of Medicine and the Glenn Biggs Institute for Alzheimer’s and Neurodegenerative Diseases, presented research that addresses a key challenge in AI-assisted brain imaging. When using an AI model, it may achieve high overall accuracy but still perform less reliably for certain populations.
Ho’s research examines ways to improve the generalizability of AI models used to classify dementia from brain imaging, particularly across racial and ethnic populations with differing representation in training data.
“We want to produce the proper probabilities for each group that we classify, but data is unbalanced for some populations,” Ho said. “The goal was to guide the model to learn the probability of each class efficiently.”
The work highlights why overall accuracy cannot establish whether an AI model’s predictions will apply to every individual. Evaluating performance across populations and improving model calibration can help researchers identify disparities and develop tools that generalize more reliably.
Adapting AI to different brain imaging datasets
The Vascular Imaging Biomarker Relationships to Alzheimer’s Disease and Related Dementias (VIBRA) project focuses on developing AI methods to measure cerebral small-vessel disease (SVD) across MRI datasets collected at different research centers. SVD, which involves damage to the brain’s small blood vessels, is associated with cognitive impairment and dementia.
VIBRA researcher and postdoctoral research fellow with the Biggs Institute Niyas Shamsudeen Kutty, PhD, said identifying imaging markers of SVD such as white matter hyperintensities, enlarged perivascular spaces, lacunes and cerebral microbleeds can be time-consuming. Differences in MRI scanners and protocols also make it difficult to apply a single AI model consistently across institutions.
Kutty and other VIBRA researchers are developing models trained on data from multiple National Institute on Aging Alzheimer’s Disease Research Centers that can be adapted to new datasets using relatively few annotated scans.
Examples presented during the conference showed that adding five to 10 annotated samples from a new dataset improved detection performance compared with using none, although errors remained.
The approach could reduce the time researchers spend labeling scans to reliably quantify SVD imaging biomarkers, helping them compare vascular brain injury and Alzheimer’s disease markers across larger populations.
Verifying AI-generated research
At the ISGC meeting, Gutiérrez discussed AI’s growing role in scientific work, including coding, data analysis and manuscript preparation. Despite its capabilities, he said, AI can sometimes produce convincing but incorrect results.
“Always challenge the output of AI models, because AI models make mistakes. Particularly, an error-free large language model (LLM) is not an engineering challenge. It’s a mathematical impossibility. LLMs will always make mistakes,” he said.
Gutiérrez advocated independent verification, including using one AI system to examine the output of another system. This approach can uncover errors that the original system misses, although it cannot eliminate every mistake. Transparent methods, reproducible workflows and independent validation remain essential.
“Since we have many agents participating in research, do we know what each agent says? Can we quantify the modes of failure of all these agents? Can we make those modes of failure visible?” he said.
Gutiérrez also encouraged researchers to understand commercial AI platforms’ confidentiality and data-use policies before submitting unpublished findings, research ideas or code. Questions about data retention, ownership and how submitted information may be used are particularly important when researchers are developing new methods or discoveries.
He said as AI assumes more research tasks, universities must also establish expectations for accountability and prepare students to evaluate evidence and recognize errors rather than accept AI-generated conclusions uncritically.
“We have a great commitment toward teaching AI, more than as a tool, as a way of thinking toward the future,” Gutiérrez said.
Better science, not just faster science
AI-assisted brain-health research has the potential to expand participation, improve prediction accuracy across populations and make imaging analysis more scalable. But with this new technology, scientists must understand where AI tools succeed, where they fail and whether AI-generated results can be trusted.
Read more:
UT San Antonio hosts international brain health conferences advancing research into dementia, stroke
UT San Antonio researchers develop model to improve machine-learning predictions